Mixed reality social interaction

ABSTRACT

Mixed reality social interactions are described. Techniques described herein include determining authentication information associated with a mixed reality display device and determining that a content item is visible in a mixed reality environment associated with the mixed reality display device. In an example, a content item may be determined to be visible based at least in part on content data indicating that the content item is owned by the mixed reality display device and/or has been shared with the mixed reality display device. The content data may also indicate an identification of a content item of the plurality of content items, an owner of the content item, and permissions associated with the content item. The techniques further describe causing a graphical representation of the content item to be presented via a display on the mixed reality display device.

RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No.62/202,614 filed on Aug. 7, 2015, the entire contents of which areincorporated herein by reference.

BACKGROUND

Computing systems can help generate new environments including virtualreality environments and/or mixed reality environments. Virtual realityis an immersive experience, which simulates physical presence in a realor imagined environment. For example, a virtual reality environment canimmerse a physical, real-world person with computer-generated graphicsin a computer-generated, virtual scene via a virtual reality displaydevice. Mixed reality, which can also be known as augmented reality, isa hybrid reality experience, which merges real worlds and virtualworlds. Mixed reality is a technology that produces mixed realityenvironments where a physical, real-world person and/or objects inphysical, real-world scenes co-exist with virtual, computer-generatedpeople and/or objects in real time. For example, a mixed realityenvironment can augment a physical, real-world scene and/or a physical,real-world person with computer-generated graphics in the physical,real-world scene viewed via a mixed reality display device.

Virtual reality environments enable multiple users to sharecomputer-generated, virtual scenes. In such examples, the multiple userscan interact with computer-generated graphics in a computer-generated,virtual scene via virtual reality display devices associated with eachof the users. However, mixed reality environments are typically limitedto single-user applications. That is, mixed reality environments are notconfigured to enable multiple users to interact with one another and/orreal objects and/or computer-generated graphics in physical, real-worldscenes at the same time.

SUMMARY

Mixed reality social interactions are described. Techniques describedherein include determining authentication information associated with amixed reality display device and determining that a content item isvisible in a mixed reality environment associated with the mixed realitydisplay device based at least in part on accessing content dataassociated with a plurality of content items and determining, based atleast in part on the authentication information and the content data,that the content item is owned by the mixed reality display deviceand/or has been shared with the mixed reality display device. Thecontent data may indicate an identification of a content item of theplurality of content items, an owner of the content item, andpermissions associated with the content item. The techniques furtherdescribe causing a graphical representation of the content item to bepresented via a display on the mixed reality display device.

It should be appreciated that the above-described subject matter can beimplemented as a computer-controlled apparatus, a computer process, acomputing system, or as an article of manufacture such as acomputer-readable storage medium. These and various other features willbe apparent from a reading of the following Detailed Description and areview of the associated drawings.

This Summary is provided to introduce a selection of techniques in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intendedthat this Summary be used to limit the scope of the claimed subjectmatter. Furthermore, the claimed subject matter is not limited toimplementations that solve any or all disadvantages noted in any part ofthis disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The Detailed Description is set forth with reference to the accompanyingfigures, in which the left-most digit of a reference number identifiesthe figure in which the reference number first appears. The use of thesame reference numbers in the same or different figures indicatessimilar or identical items or features.

FIG. 1 is a schematic diagram showing an example environment forenabling two or more users in a mixed reality environment to interactwith one another and/or with virtual content that is presented in themixed reality environment.

FIG. 2 is a schematic diagram showing an example of a head mounted mixedreality display device.

FIG. 3 is a schematic diagram showing an example of a first view of amixed reality environment wherein two or more users can interact withone another and/or with virtual content that is presented in the mixedreality environment.

FIG. 4 is a schematic diagram showing an example of a second view of amixed reality environment wherein two or more users can interact withone another and/or with virtual content that is presented in the mixedreality environment.

FIG. 5 is a schematic diagram showing an example environment forenabling two or more users in a mixed reality environment to interactwith one another and/or with virtual content that is presented in themixed reality environment.

FIG. 6 is a flow diagram that illustrates an example process to causevirtual content to be presented in the mixed reality environment.

FIG. 7 is a flow diagram that illustrates an example process to causevirtual content to be presented in the mixed reality environment indifferent modes (e.g., presenter mode or sharing mode).

FIG. 8 is a schematic diagram showing an example environment forenabling two or more users in a mixed reality environment to interactwith one another and/or with virtual content that is presented in themixed reality environment.

FIG. 9 is a flow diagram that illustrates an example process to causevirtual content to be presented in the mixed reality environment.

FIG. 10 is a schematic diagram showing an example environment forenabling two or more users in a mixed reality environment to interactwith one another and/or with virtual content that is presented in themixed reality environment.

FIG. 11 is a flow diagram that illustrates an example process to causethe visibility of virtual content to be modified in a mixed realityenvironment.

FIG. 12 is a flow diagram that illustrates an example process to causean interaction associated with the virtual content to be performed viaone or more devices in a mixed reality environment.

DETAILED DESCRIPTION

This disclosure describes techniques for enabling two or more users in amixed reality environment to interact with one another and/or withvirtual content that is presented in the mixed reality environment. Thetechniques described herein can enhance mixed reality socialinteractions between users in mixed reality environments. In at leastone example, the techniques are directed to mixed reality socialinteractions between two or more users who are physically located in asame real scene, as described below, and the real scene is unmarked(i.e., lacking predetermined and/or machine vision-specific markings fordirecting interactions between the two or more users). The techniquesdescribed herein can have various applications, including but notlimited to, enabling users that are located in a same real scene toshare virtual content and/or interact with the virtual content in amixed reality environment via mixed reality display devices. Thetechniques described herein enable enhanced user interfaces to bepresented on displays of mixed reality devices thereby enhancing mixedreality social interactions between users and the mixed realityexperience.

For the purposes of this discussion, physical, real-world objects (“realobjects”) or physical, real-world people (“real people” and/or “realperson”) describe objects or people, respectively, that physically existin a physical, real-world scene (“real scene”) associated with a mixedreality display. Real objects and/or real people can move in and out ofa field of view based on movement patterns of the real objects and/ormovement of a user and/or user device. Virtual, computer-generatedcontent (“virtual content” and/or “content items”) can describe contentthat is generated by one or more computing devices to supplement thereal scene in a user's field of view. In at least one example, virtualcontent can include one or more pixels each having a respective color orbrightness that are collectively presented on a display such torepresent a person, object, etc. that is not physically present in areal scene. That is, in at least one example, virtual content caninclude graphics that are representative of objects (“virtual objects”),people (“virtual people” and/or “virtual person”), biometric data,effects, etc. Virtual content can include two dimensional graphics,three dimensional objects, content associated with applications, etc.Virtual content can be rendered into the mixed reality environment viatechniques described herein. In additional and/or alternative examples,virtual content can include computer-generated content such as sound,video, global positioning system (GPS), etc.

Mixed reality experiences offer different opportunities to affectself-perception and new ways for communication to occur. The techniquesdescribed herein enable users to interact with one another and/or withvirtual content in mixed reality environments using mixed realitydevices. In at least one example, the techniques described herein canenable conversational partners to share virtual content and/or interactwith virtual content in mixed reality environments. While the techniquesdescribed herein are directed to mixed reality environments, asdescribed above, mixed reality may also be known as augmented reality.Accordingly, the techniques described herein should not be construed toexclude augmented reality environments.

Illustrative Environments

FIG. 1 is a schematic diagram showing an example environment 100 forenabling two or more users in a mixed reality environment to interactwith one another and with virtual content that is presented in the mixedreality environment. More particularly, the example environment 100 caninclude a service provider 102, one or more networks 104, one or moreusers 106 (e.g., user 106A, user 106B, user 106C, etc.) and one or moredevices 108 (e.g., device 108A, device 108B, device 108C, etc.)associated with the one or more users 106 (e.g., user 106A, user 106B,user 106C, etc.).

The service provider 102 can be any entity, server(s), platform,console, computer, etc., that facilitates two or more users 106interacting in a mixed reality environment to enable individual users(e.g., user 106A, user 106B, and/or user 106C) of the two or more users106 to interact with one another and/or with virtual content in themixed reality environment. The service provider 102 can be implementedin a non-distributed computing environment or can be implemented in adistributed computing environment, possibly by running some modules ondevices 108 or other remotely located devices. As shown, the serviceprovider 102 can include one or more server(s) 110, which can includeone or more processing unit(s) (e.g., processor(s) 112) andcomputer-readable media 114, such as memory. In various examples, theservice provider 102 can access, receive, and/or determineauthentication data from a device (e.g., device 108A), access contentdata associated with virtual content items, send rendering dataassociated with individual virtual content items to the device (e.g.,device 108A), and cause the individual virtual content items to bepresented on a display associated with the device (e.g., device 108A).For the purpose of this discussion, rendering data may includeinstructions for rendering a graphical representation of a virtualcontent item via a display of a device (e.g., device 108A). Forinstance, the rendering data may include instructions describing thegeometry, viewpoint, texture, lighting, shading, etc. associated with avirtual content item. In some examples, the service provider 102 maysend rendering data to devices 108 and the devices 108 can render thegraphical representations via displays associated with the devices. Inother examples, as described below, the service provider 102 may renderframes and may send the frames to the devices 108 for presentation viathe displays.

In some examples, the service provider 102 can receive frame requestsfrom a device (e.g., device 108A) and can send frame messages to thedevice (e.g., device 108A) to mitigate latency caused by movement thatoccurs between sending the frame requests to the service provider 102and receiving frame messages at and/or rendering corresponding framesvia the device (e.g., device 108A). In at least one example, the serviceprovider 102 can receive requests from individual devices (e.g., device108A, device 108B, device 108C, etc.) of the one or more devices 108associated with sharing virtual content items with other devices 108(e.g., a request to view and/or access a virtual content items) and/orrequests for performing interactions on the virtual content items, andthe service provider 102 can synchronize communications and/or contentrendering between the devices 108 to ensure that the virtual contentitems and interactions directed to the virtual content items arepresented to corresponding users 106 at a substantially same time sothat each of the users 106 has a same experience.

In some examples, the networks 104 can be any type of network known inthe art, such as the Internet. Moreover, the devices 108 cancommunicatively couple to the networks 104 in any manner, such as by aglobal or local wired or wireless connection (e.g., local area network(LAN), intranet, Bluetooth, etc.). The networks 104 can facilitatecommunication between the server(s) 110 and the devices 108 associatedwith the one or more users 106.

Examples support scenarios where device(s) that can be included in theone or more server(s) 110 can include one or more computing devices thatoperate in a cluster or other clustered configuration to shareresources, balance load, increase performance, provide fail-over supportor redundancy, or for other purposes. Device(s) included in the one ormore server(s) 110 can represent, but are not limited to, desktopcomputers, server computers, web-server computers, personal computers,mobile computers, laptop computers, tablet computers, wearablecomputers, implanted computing devices, telecommunication devices,automotive computers, network enabled televisions, thin clients,terminals, game consoles, gaming devices, work stations, media players,digital video recorders (DVRs), set-top boxes, cameras, integratedcomponents for inclusion in a computing device, appliances, or any othersort of computing device.

Device(s) that can be included in the one or more server(s) 110 caninclude any type of computing device having one or more processingunit(s) (e.g., processor(s) 112) operably connected to computer-readablemedia 114 such as via a bus, which in some instances can include one ormore of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCIbus, and any variety of local, peripheral, and/or independent buses.Executable instructions stored on computer-readable media 114 caninclude, for example, an input module 116, a content database 118, acontent management module 120, a frame rendering module 122, apositioning module 124, and one or more applications 126, and othermodules, programs, or applications that are loadable and executable bythe processor(s) 112.

Alternatively, or in addition, the functionality described herein can beperformed, at least in part, by one or more hardware logic componentssuch as accelerators. For example, and without limitation, illustrativetypes of hardware logic components that can be used includeField-programmable Gate Arrays (FPGAs), Application-specific IntegratedCircuits (ASICs), Application-specific Standard Products (ASSPs),System-on-a-chip systems (SOCs), Complex Programmable Logic Devices(CPLDs), etc. Device(s) that can be included in the one or moreserver(s) 110 can further include one or more input/output (I/O)interface(s) coupled to the bus to allow device(s) to communicate withother devices such as input peripheral devices (e.g., a keyboard, amouse, a pen, a game controller, a voice input device, a touch inputdevice, gestural input device, a tracking device, a mapping device, animage camera, a time-of-flight (TOF) camera, a depth sensor, aphysiological sensor, and the like) and/or output peripheral devices(e.g., a display, a printer, audio speakers, a haptic output, and thelike). Such network interface(s) can include one or more networkinterface controllers (NICs) or other types of transceiver devices tosend and receive communications over a network. For simplicity, somecomponents are omitted from the illustrated environment.

Processing unit(s) (e.g., processor(s) 112) can represent, for example,a CPU-type processing unit, a GPU-type processing unit, an HPU-typeprocessing unit, a field-programmable gate array (FPGA), another classof digital signal processor (DSP), or other hardware logic componentsthat can, in some instances, be driven by a CPU. For example, andwithout limitation, illustrative types of hardware logic components thatcan be used include Application-Specific Integrated Circuits (ASICs),Application-Specific Standard Products (ASSPs), System-on-a-chip systems(SOCs), Complex Programmable Logic Devices (CPLDs), etc. In variousexamples, the processing unit(s) (e.g., processor(s) 112) can executeone or more modules and/or processes to cause the server(s) 110 toperform a variety of functions, as set forth above and explained infurther detail in the following disclosure. Additionally, each of theprocessing unit(s) (e.g., processor(s) 112) can possess its own localmemory, which also can store program modules, program data, and/or oneor more operating systems.

In at least one configuration, the computer-readable media 114 of theserver(s) 110 can include components that facilitate interaction betweenthe service provider 102 and the one or more devices 108. The componentscan represent pieces of code executing on a computing device. Forexample, the computer-readable media 114 can include the input module116, the content database 118, the content management module 120, theframe rendering module 122, the positioning module 124, and the one ormore applications 126, etc. In at least some examples, the modules canbe implemented as computer-readable instructions, various datastructures, and so forth via at least one processing unit(s) (e.g.,processor(s) 112) to enable two or more users 106 in a mixed realityenvironment to interact with one another and with virtual content thatis presented in the mixed reality environment. Functionality to performthese operations can be included in multiple devices or a single device.

Depending on the exact configuration and type of the server(s) 110, thecomputer-readable media 114 can include computer storage media and/orcommunication media. Computer storage media can include volatile memory,nonvolatile memory, and/or other persistent and/or auxiliary computerstorage media, removable and non-removable computer storage mediaimplemented in any method or technology for storage of information suchas computer readable instructions, data structures, program modules, orother data. Computer memory is an example of computer storage media.Thus, computer storage media includes tangible and/or physical forms ofmedia included in a device and/or hardware component that is part of adevice or external to a device, including but not limited torandom-access memory (RAM), static random-access memory (SRAM), dynamicrandom-access memory (DRAM), phase change memory (PRAM), read-onlymemory (ROM), erasable programmable read-only memory (EPROM),electrically erasable programmable read-only memory (EEPROM), flashmemory, compact disc read-only memory (CD-ROM), digital versatile disks(DVDs), optical cards or other optical storage media, miniature harddrives, memory cards, magnetic cassettes, magnetic tape, magnetic diskstorage, magnetic cards or other magnetic storage devices or media,solid-state memory devices, storage arrays, network attached storage,storage area networks, hosted computer storage or any other storagememory, storage device, and/or storage medium that can be used to storeand maintain information for access by a computing device.

In contrast, communication media can embody computer readableinstructions, data structures, program modules, or other data in amodulated data signal, such as a carrier wave, or other transmissionmechanism. The term “modulated data signal” means a signal that has oneor more of its characteristics set or changed in such a manner as toencode information in the signal. Such signals or carrier waves, etc.can be propagated on wired media such as a wired network or direct-wiredconnection, and/or wireless media such as acoustic, RF, infrared andother wireless media. As defined herein, computer storage media does notinclude communication media. That is, computer storage media does notinclude communications media consisting solely of a modulated datasignal, a carrier wave, or a propagated signal, per se.

The input module 116 is configured to receive input from one or moredevices 108 (e.g., device 108A, device 108B, device 108C, etc.) eachcorresponding to a user (e.g., user 106A, user 106B, user 106C, etc.).In at least one example, the input module 116 can access, receive,and/or determine authentication data from a device (e.g., device 108A).The authentication data can correspond to a user identification andpassword associated with a user (e.g., user 106A) associated with thedevice (e.g., device 108A), biometric identification associated with auser (e.g., user 106A) associated with the device (e.g., device 108A),etc. In at least one example, the authentication data can be leveragedto determine presence of corresponding devices 108 in a mixed realityenvironment. For the purpose of this discussion, presence may indicatethat a device (e.g., device 108A) is located in and/or interacting withother devices (e.g., device 108B, device 108C, etc.) and/or virtualcontent in a mixed reality environment.

In additional and/or alternative examples, the authentication data canbe utilized to determine virtual content items that are available to theuser (e.g., user 106A) and the user's (e.g., user 106A) permissionscorresponding to viewing and/or interacting with each of the virtualcontent items. In at least one example, the authentication data can beutilized for causing virtual content items to be presented in a samemixed reality environment where a user (e.g. user 106A) previously leftthe virtual content item and in a same position where the user (e.g.,user 106A) previously left the virtual content item (e.g., if a user(e.g., user 106A) removes his or her head mounted display device (e.g.,device 108A), turns off his or her device (e.g., device 108A), etc.).

The content database 118 is configured to store content data associatedwith virtual content. Content data associated with the individualvirtual content items can be stored in the content database 118. Eachindividual virtual content item can be associated with data indicatingan owner identification, a content identification, and permissions(i.e., permissions data). Data associated with an owner of a virtualcontent item may identify a user (e.g., user 106A, user 106B, user 106C,etc.) that generated and/or has control over the permissions associatedwith a virtual content item. That is, an owner of a virtual content itemcan correspond to a user (e.g., user 106A, user 106B, user 106C, etc.)that generated and/or has control over the permissions associated withthe virtual content item. Content identification can correspond to dataindicating the content associated with the virtual content item.Permissions data can include information indicating which users 106and/or corresponding devices 108 have permission to view and/or interactwith the virtual content in the mixed reality environment (i.e., whichusers 106 the owner has shared the virtual content with). For instance,the permission data can reflect whether a virtual content item ispublic, private, visible by some devices (e.g., device 108A, device108B, and/or device 108C), etc. Additionally and/or alternatively, thepermissions data can indicate which interactions particular users 106can perform and/or which interactions particular users 106 areprohibited from performing In some examples, the permissions data can bebased on input from the owner of the corresponding virtual content item,as described below.

In at least one example, the user (e.g., user 106A) associated with adevice (e.g., device 108A) that initially requests the virtual contentitem can be the owner of the virtual content item such that he or shecan modify the permissions associated with the virtual content item. Inat least one example, the owner of the virtual content item candetermine which other users (e.g., user 106B and/or user 106C) can viewthe virtual content item (i.e., whether the virtual content item isvisible to the other users 106). For instance, in an example, an ownerof a virtual content item can utilize a menu (e.g., a dropdown menu, aradial menu, etc.) or other mechanisms to share the virtual content itemwith all users 106 in a same mixed reality environment (i.e., make thevirtual content item public), share the virtual content item with someusers (e.g., user 106A, user 106B, and/or user 106C) in the same mixedreality environment, or not share the virtual content item with anyother users 106 (i.e., make the virtual content item private). That is,in some examples, the owner of the virtual content item can determinewhether a virtual content item is visible or not visible via otherdevices 108. In other examples, the owner of the virtual content itemcan determine which other users (e.g., user 106B and/or user 106C) caninteract with the virtual content item via corresponding devices (e.g.,device 108B and/or device 108C) and/or which interactions are permitted.

The content management module 120 manages the ownership of virtualcontent items and can leverage the permissions data to determine whichof the other users (e.g., user 106B and/or user 106C) and/orcorresponding devices (e.g., device 106B and/or user 106C) havepermission to view individual virtual content items and/or interact withindividual virtual content items. That is, the content management module120 may access the content data to determine devices 108 with which acontent item has been shared and/or interactions available for each ofthe devices 108. As described above, the content data may includepermissions data which indicates whether a content item is public,private, or has been shared with one or more devices (e.g., device 108B,device 108C, etc.) and/or interactions available for each of the devices108.

In various examples, the frame rendering module 122 can receive framerequest messages from a requesting device (e.g., device 108A) of the oneor more devices 108. Frame request messages can include, but are notlimited to, pose information associated with each eye of a user (e.g.,user 106A), a timestamp, a desired resolution, and a desired field ofview. Pose information can include a position and a rotation relative toa common coordinate system (i.e., a coordinate system that isconsistently defined for both the device (e.g., device 108A) and theservice provider 102), which for the purpose of this discussion, may bereferred to as the worldspace coordinate system. A time stamp mayrepresent a time in which the frame request message was generated and/orsent. A desired resolution may be a desired level of detail associatedwith rendered virtual content (i.e., a higher resolution amounts to moredetail in the virtual content). In some examples, resolution candescribe a pixel count in a digital image. A desired field of view maydescribe an extent to which the observable world is desired to be seenat any given time through a display of a mixed reality display device(e.g., device 108A, device 108B, device 108C, etc.). In some examples,field of view may describe an angle of view.

The frame request message can be processed by the frame rendering module122 to enable virtual content to be rendered from a particular user'spoint of view (e.g., user 106A). The frame rendering module 122 maygenerate a frame message responsive to a frame request message. Theresulting frame message can include a same timestamp which was sent inthe associated frame request message, the determined resolution, thedetermined field of view, the pose of each eye as sent in the associatedframe request message, and the render distance. In some examples, theframe rendering module 122 can be configured to render stereo images(one image per eye of a user (e.g., user 106A)) for each frame requestmessage. The stereo images may represent frames. A first image of thestereo images can correspond to the left eye of a user (e.g., user 106A)and a second image of the stereo images can correspond to a right eye ofa user (e.g., user 106A). In at least one example, a frame message mayinclude rendered stereo images. In some examples, the frame renderingmodule 122 can render a mixed reality scene at a different resolution orfield of view than the requested desired values. The resultingresolution and/or field of view may be associated with the framemessage, described above. The frame rendering module 122 can send theframe message to the requesting device (e.g., device 108A).

The requesting device (e.g., device 108A) can receive the frame message,process the received frame message, and render the stereo images as twoquads, or other virtual surfaces, positioned in worldspace in front of avirtual stereo camera associated with the requesting device (e.g.,device 108A). In an example, the left stereo image can be textured ontothe left quad and the right stereo image can be textured onto the rightquad. In such an example, the left quad may be rendered by a left cameraassociated with a device (e.g., device 108A) and the right quad may berendered by a right camera associated with a device (e.g., device 108A).Each quad can be positioned in worldspace in front of each eye asspecified in the frame message, such that each quad's normal vector isaligned with the eye direction vector. Each quad can be sized such thatit can fill the frustum defined by each eye in the received framemessage, which can be defined by the combination of the determined fieldof view and the eye pose information in the frame message. Therequesting device (e.g., device 108A), can continue to render both theleft and right quads as the user (e.g., user 106A) moves about inworldspace (with the quads fixed in worldspace), until the next framerequest message is sent to the frame rendering module 122 and theresponsive frame message is received by the requesting device (e.g.,device 108A). Based at least in part on receiving the next framemessage, the left and right quads can be repositioned and retextured asdescribed according to the data in the frame message (e.g., a sametimestamp which was sent in the associated frame request message, thedetermined resolution, the determined field of view, the pose of eacheye as sent in the associated frame request message, and the renderdistance).

Before the next frame message is received by the requesting device(e.g., 108A), any movement of the user (e.g., user 106A), andcorresponding device (e.g., device 108A), relative to the left and rightquads (which are fixed in worldspace) can appear as a corresponding andopposite movement of the left and right quads in screenspace (e.g.,relative to the screen). For the purpose of this discussion, screenspacecan represent the space defined by the display 204 associated with adevice (e.g., device 108A). For each frame message, there can be aninfinite number of possible valid positions and sizes for the left andright quads defined by a proportional relationship between theworldspace distance from each quad to each eye and the worldspace sizeof each quad (i.e., the further away these quads are, the larger theymay be in order to fill each eye frustum appropriately). The amount ofmovement in screenspace can be proportionately affected by the distanceat which these quads are positioned relative to the user (e.g., user106A) (i.e., the parallax effect).

To create more natural movement of these quads in screenspace (betweenframe messages) the distance of these quads (from their associated eyepositions) can be determined by using a heuristic to approximate anappropriate distance of the quads. An example of a heuristic can be tocalculate the average distance of each virtual object which is visiblein the rendered frame. Another example can be to calculate the averagedistance of each pixel that is visible in the frame rendering module122. An additional and/or alternative example can be to calculate thedistance of the most salient object (or the most salient pixels) in thescene (as determined by any number of factors, including gaze tracking).The frame rendering module 122 can use any of these (or any other)heuristics to calculate a render distance for each frame, which can alsobe sent in each frame message. This render distance can then be used todefine a specific position and size at which the requesting device(e.g., device 108A) can position the left and right quads.

In at least one example, to calculate an average pixel distance, theframe rendering module 122 can render a depth buffer for each frame froma center eye anchor (i.e., the center between both eyes of a user (e.g.,user 106A)). In the at least one example, the depth buffer can berendered using a shader that outputs the pixel depth mapped to a valuebetween 0 and 1 (linearly or otherwise), with 0 being the camera's nearplane, and 1 being the camera's far plane. As a non-limiting example, adepth value can be encoded either into one (8-bit) channel of the outputbuffer, such that the depth value is encoded with a resolution of 255values (1 byte), or alternatively all four channels in a 32-bit buffercan be leveraged to encode a 32-bit floating point value representingthe same depth value (between 0 and 1) at 32-bit precision for eachpixel. In the non-limiting example, the resulting depth buffer values(once decoded into a standard 32-bit floating point representation) canbe used to determine the worldspace distance between each pixel and thecamera which was used to render the depth buffer. In the non-limitingexample, the worldspace distance for each pixel is determined bysubtracting the near plane distance from the far plane distance,multiplying that difference by the pixel's depth value, and then addingthe near plane distance to the result. The frame rendering module 122can then calculate an average pixel distance by averaging the worldspacedistance of each pixel. This average pixel distance can be included inthe frame message as the render distance.

In some examples, the frame rendering module 122 may send the depthbuffer data in the frame message to the requesting device (e.g., device108A) and a parallax shader can be used by the requesting device (e.g.,device 108A) to approximate movement of the user (e.g., user 106A). Insuch examples, the frame message may include additional and/oralternative data (e.g., the depth buffer, either for each eye, or forthe center eye anchor), and the rendering module 132 may render thevirtual content items in the mixed reality environment. In suchexamples, the frame rendering module 122 may not calculate the averagepixel distance and/or a saliency map, as described above.

In at least some examples, the frame rendering module 122 may access thecontent data to determine which virtual content items a user (e.g., user106A) has open and/or which virtual content items the user (e.g., user106A) has shared with other users (e.g., user 106B and/or user 106C).

The positioning module 124 can send instructions associated withrendering virtual content on a display of a device (e.g., device 108A)to the device (e.g., device 108A). That is, the positioning module 124can send instructions associated with a position and/or placement ofvirtual content in a mixed reality environment. The instructions can bedetermined by the content data, and in some examples, may be associatedwith the rendering data, described below.

Applications (e.g., application(s) 126) are created by programmers tofulfill specific tasks. For example, applications (e.g., application(s)126) can provide utility, entertainment, educational, and/orproductivity functionalities to users 106 of devices 108. Applications(e.g., application(s) 126) can be built into a device (e.g.,telecommunication, text message, clock, camera, etc.) or can becustomized (e.g., games, news, transportation schedules, onlineshopping, etc.). Application(s) 126 can provide conversational partners(e.g., two or more users 106) various functionalities, including but notlimited to, sharing and/or interacting with virtual content items in amixed reality environment. In at least some examples, the virtualcontent items can be applications and/or can be associated with theapplications.

In some examples, the one or more users 106 can operate correspondingdevices 108 (e.g., user devices) to perform various functions associatedwith the devices 108. Device(s) 108 can represent a diverse variety ofdevice types and are not limited to any particular type of device.Examples of device(s) 108 can include but are not limited to mobilecomputers, embedded computers, or combinations thereof. Example mobilecomputers can include laptop computers, tablet computers, wearablecomputers, implanted computing devices, telecommunication devices,automotive computers, portable gaming devices, media players, cameras,or the like. Example embedded computers can include network enabledtelevisions, integrated components for inclusion in a computing device,appliances, microcontrollers, digital signal processors, or any othersort of processing device, or the like. In at least one example, thedevices 108 can include mixed reality devices (e.g., CANON® MREAL®System, MICROSOFT® HOLOLENS®, etc.). Mixed reality devices can includeone or more sensors and a mixed reality display, as described below inthe context of FIG. 2. In FIG. 1, device 108A, device 108B, and device108C are wearable computers (e.g., head mount devices); however, thedevices 108 can be any other device as described above. In at least oneexample, the devices 108 can be untethered such that they are notphysically connected to external devices. However, the devices 108 canbe communicatively coupled to external devices, as described herein.

Device(s) 108 can include one or more input/output (I/O) interface(s)coupled to the bus to allow device(s) to communicate with other devicessuch as input peripheral devices (e.g., a keyboard, a mouse, a pen, agame controller, a voice input device, a touch input device, gesturalinput device, a tracking device, a mapping device, an image camera, adepth sensor, a physiological sensor, and the like) and/or outputperipheral devices (e.g., a display, a printer, audio speakers, a hapticoutput, and the like). As described above, in some examples, the I/Odevices can be integrated into the one or more server(s) 110 and/orother machines and/or devices 108. In other examples, the one or moreinput peripheral devices can be communicatively coupled to the one ormore server(s) 110 and/or other machines and/or devices 108. The one ormore input peripheral devices can be associated with a single device(e.g., MICROSOFT® KINECT®, INTEL® Perceptual Computing SDK 2013, LEAPMOTION®, etc.) or separate devices.

FIG. 2 is a schematic diagram showing an example of a head mounted mixedreality display device 200. As illustrated in FIG. 2, the head mountedmixed reality display device 200 can include one or more sensors 202 anda display 204. The one or more sensors 202 can reconstruct the realscene in which the one or more users 106 are physically located andtrack real people and/or objects within the real scene. The one or moresensors 202 can include cameras and/or sensors. The cameras can includeimage cameras, stereoscopic cameras, trulight cameras, etc. The sensorscan include depth sensors, color sensors, acoustic sensors, opticalsensors, pattern sensors, gravity sensors, etc. The cameras and/orsensors can output streams of data in substantially real time. The datacan include moving image data and/or still image data (e.g., trackingdata) representative of movement of real people and/or real objects in areal scene that is observable by the cameras and/or sensors.Additionally, the data can include depth data.

Tracking devices can output the moving image data and/or still imagedata (e.g., tracking data) representative of movement of real peopleand/or real objects in a real scene. Tracking devices can includeoptical tracking devices (e.g., VICON®, OPTITRACK®), magnetic trackingdevices, acoustic tracking devices, gyroscopic tracking devices,mechanical tracking systems, depth cameras (e.g., KINECT®, INTEL®RealSense, etc.), inertial sensors (e.g., INTERSENSE®, XSENS, etc.),combinations of the foregoing, etc. The tracking devices can outputstreams of volumetric data, skeletal data, perspective data, etc. insubstantially real time. The streams of volumetric data, skeletal data,perspective data, etc. can be received by the input module 116 insubstantially real time. Volumetric data can correspond to a volume ofspace occupied by a body of a user (e.g., user 106A, user 106B, or user106C). Skeletal data can correspond to data used to approximate askeleton, in some examples, corresponding to a body of a user (e.g.,user 106A, user 106B, or user 106C), and track the movement of theskeleton over time. The skeleton corresponding to the body of the user(e.g., user 106A, user 106B, or user 106C) can include an array of nodesthat correspond to a plurality of human joints (e.g., elbow, knee, hip,etc.) that are connected to represent a human body. Perspective data cancorrespond to data collected from two or more perspectives that can beused to determine an outline of a body of a user (e.g., user 106A, user106B, or user 106C) from a particular perspective.

Combinations of the volumetric data, the skeletal data, and theperspective data can be used to determine body representationscorresponding to users 106. The body representations can approximate abody shape of a user (e.g., user 106A, user 106B, or user 106C). Thatis, volumetric data associated with a particular user (e.g., user 106A),skeletal data associated with a particular user (e.g., user 106A), andperspective data associated with a particular user (e.g., user 106A) canbe used to determine a body representation that represents theparticular user (e.g., user 106A). The body representations can be usedby the rendering module 132 to determine where to render virtual contentin the three dimensional coordinate system (e.g. worldspace)corresponding to the real space where the particular user (e.g., user106A) is physically located.

The depth data can represent distances between real objects in a realscene observable by sensors and/or cameras and the sensors and/orcameras. The depth data can be based at least in part on infrared (IR)data, trulight data, stereoscopic data, light and/or pattern projectiondata, gravity data, acoustic data, etc. In at least one example, thestream of depth data can be derived from IR sensors (e.g., time offlight, etc.) and can be represented as a point cloud reflective of thereal scene. The point cloud can represent a set of data points or depthpixels associated with surfaces of real objects and/or the real sceneconfigured in a three dimensional coordinate system (e.g., worldspace).The depth pixels can be mapped into a grid. The grid of depth pixels canindicate a distance between real objects in the real scene and thecameras and/or sensors. The grid of depth pixels that correspond to thevolume of space that is observable from the cameras and/or sensors canbe called a depth space. The depth space can be utilized by therendering module 132 (in the devices 108) for determining how to rendervirtual content in the mixed reality display.

In some examples, the one or more sensors 202 can be integrated into thehead mounted mixed reality display device 200 and/or devices 108. Insuch examples, the one or more sensors 202 correspond to inside-outsensing sensors; that is, sensors that capture information from a firstperson perspective. In additional or alternative examples, the one ormore sensors can be external to the head mounted mixed reality displaydevice 200 and/or devices 108. In such examples, the one or more sensors202 can be arranged in a room (e.g., placed in various positionsthroughout the room), associated with a device, etc. Such sensors cancorrespond to outside-in sensing sensors; that is, sensors that captureinformation from a third person perspective. In yet another example, thesensors can be external to the head mounted mixed reality display device200 but can be associated with one or more wearable devices configuredto collect data associated with the user (e.g., user 106A, user 106B, oruser 106C).

The display 204 can present visual content to the one or more users 106in a mixed reality environment. In some examples, the display 204 canpresent the mixed reality environment to a user (e.g., user 106A) in aspatial region that occupies an area that is substantially coextensivewith the user's (e.g., user 106A) actual field of vision. In otherexamples, the display 204 can present the mixed reality environment tothe user (e.g., user 106A) in a spatial region that occupies a lesserportion of a user's (e.g., user 106A) actual field of vision. Thedisplay 204 can include a transparent display that enables a user (e.g.,user 106A) to view the real scene where he or she is physically located.Transparent displays can include optical see-through displays where theuser (e.g., user 106A) sees the real scene he or she is physicallypresent in directly, video see-through displays where the user (e.g.,user 106A) observes the real scene in a video image acquired from amounted camera, etc. The display 204 can present the virtual content tothe user (e.g., user 106A) such that the virtual content augments thereal scene where the user (e.g., user 106A) is physically located withinthe spatial region.

The virtual content can appear differently to different users (e.g.,user 106A, user 106B, and/or user 106C) based on the users' perspectivesand/or the location of the corresponding devices (e.g., device 108A,device 108B, and/or device 108C). For instance, the size of a virtualcontent item can be different based on a proximity of a user (e.g., user106A, user 106B, and/or user 106C) and/or device (e.g., device 108A,device 108B, and/or device 108C) to the virtual content item.Additionally or alternatively, the shape of the virtual content item canbe different based on the vantage point of a user (e.g., user 106A, user106B, and/or user 106C) and/or device (e.g., device 108A, device 108B,and/or device 108C). For instance, a virtual content item can have afirst shape when a user (e.g., user 106A, user 106B, and/or user 106C)and/or device (e.g., device 108A, device 108B, and/or device 108C) islooking at the virtual content item straight on and can have a secondshape when a user (e.g., user 106A, user 106B, and/or user 106C) and/ordevice (e.g., device 108A, device 108B, and/or device 108C) is lookingat the virtual item from the side.

The devices 108 can include one or more processing unit(s) (e.g.,processor(s) 128), computer-readable media 130, at least including arendering module 132 and one or more applications 134. The one or moreprocessing unit(s) (e.g., processor(s) 128) can represent same unitsand/or perform same functions as processor(s) 112, described above.Computer-readable media 130 can represent computer-readable media 114 asdescribed above. Computer-readable media 130 can include components thatfacilitate interaction between the service provider 102 and the one ormore devices 108. The components can represent pieces of code executingon a computing device, as described above. Computer-readable media 128can include at least a rendering module 132. The rendering module 132can receive content data from the service provider 102 and can rendervirtual content items on the display 204 of the device (e.g., device108A, device 108B, or device 108C). In at least one example, therendering module 132 can leverage a standard graphics rendering pipelinefor rendering virtual content on the display 204. In some examples, therendering module 132 can receive previously rendered frames (e.g.,associated with frame messages) from the service provider 102 to correctfor potential latency and/or render correct perspectives based on theposition of the user (e.g., user 106A) in worldspace. In other examples,the rendering module 132 may receive rendering data for rendering thevirtual content items locally. Application(s) 134 can correspond to sameapplications as application(s) 128 or different applications.

Example Mixed Reality User Interfaces

FIG. 3 is a schematic diagram showing an example of a first view 300(e.g., a front view) of a mixed reality environment wherein two or moreusers (e.g., user 106A and user 106B) can interact with one anotherand/or with virtual content that is presented in the mixed realityenvironment. The area depicted in the dashed lines corresponds to a realscene 302 in which the first user (e.g., user 106A) and a second user(e.g., user 106B) are physically present. In some examples, the firstuser (e.g., user 106A) or the second user (e.g., user 106B) may beremotely located and can be virtually present in the mixed realityenvironment (e.g., as an avatar, a reconstructed three dimensional modelthat has been captured using various sensors and/or cameras (e.g.,KINECT® or TOF camera)). That is, the device (e.g., device 108A)corresponding to one of the users (e.g., user 106A) may receivestreaming data to render the remotely located user (e.g., user 106B) inthe mixed reality environment presented by the device (e.g., device108A). The area depicted in the solid black line corresponds to thespatial region 304 in which the mixed reality environment is visible tothe first user (e.g., user 106A) via a display 204 of a correspondingdevice (e.g., device 108A). As described above, in some examples, thespatial region can occupy an area that is substantially coextensive witha user's (e.g., user 106A) actual field of vision and in other examples,the spatial region can occupy a lesser portion of a user's (e.g., user106A) actual field of vision. Additional users (e.g., user 106C, etc.)can be present but not visible in the spatial region 304.

In the example of FIG. 3, the first user (e.g., user 106A) can beviewing various virtual content items in his mixed reality environmentvia a display 204 of his device (e.g., device 108A). As described below,the first user (e.g., user 106A) can have authenticated his device(e.g., device 108A). Based at least in part on authenticating his device(e.g., device 108A), the rendering module 132 associated with his device(e.g., device 108A) can render the various virtual content items on thedisplay 204 corresponding to his device (e.g., device 108A). As anon-limiting example, the first user (e.g., user 106A) can view agraphical representation of a globe 306, a dog 308, a drone 310, a hotair balloon 312, and a calendar 314. The graphical representations areshown in thick black lines to show that they represent virtual content.

In some examples, the first user (e.g., user 106A) can be the owner ofeach virtual content item. That is, the first user (e.g., user 106A) candetermine whether to share individual virtual content items with otherusers (e.g., user 106B and/or user 106C) in the mixed realityenvironment, which of the other users (e.g., user 106B and/or user 106C)he wants to share individual virtual content items with, or whether tokeep individual virtual content items private (i.e., not share withother users). In the example in FIG. 3, the first user (e.g., user 106A)shared the drone 310 with the second user (e.g., user 106B). The seconduser (e.g., user 106B) can view the drone 310 and can interact with thedrone 310 until the first user (e.g., user 106A) decides to make thedrone 310 private. Based at least in part on the first user (e.g., user106A) making the drone 310 private, the second user (e.g., user 106B)cannot view the drone 310 in the mixed reality environment.

In other examples, one of the other users (e.g., user 106B and/or user106C) can be the owner of the virtual content item. Accordingly, some ofthe virtual content items can be visible to the first user (e.g., user106A) based at least in part on the owner of the virtual content itemscontinuing to share the virtual content items with the first user (e.g.,user 106A). However, once the owner makes the virtual content itemprivate and/or no longer shares the virtual content item with the firstuser (e.g., user 106A), the virtual content item may no longer bevisible to the first user (e.g., user 106A).

FIG. 4 is a schematic diagram showing an example of a second view 400(e.g., a right side view) of a mixed reality environment wherein two ormore users (e.g., user 106A, user 106B, and/or user 106C) can interactwith one another and/or with virtual content that is presented in themixed reality environment. The area depicted in the dashed linescorresponds to the real scene 302 in which the first user (e.g., user106A), the second user (e.g., user 106B), and a third user (e.g., user106C) are physically present. The area depicted in the solid black linecorresponds to the spatial region 304 in which the mixed realityenvironment is visible to a third user (e.g., user 106C) via a display204 of a corresponding device (e.g., device 108C). As described above,in some examples, the spatial region can occupy an area that issubstantially coextensive with a user's (e.g., user 106C) actual fieldof vision and in other examples, the spatial region can occupy a lesserportion of a user's (e.g., user 106C) actual field of vision. Asdescribed above, FIG. 4 depicts a different view of the real scene 302in FIG. 3. FIG. 4 depicts a right side view. That is, the third user(e.g., user 106C) is located on the right side of the room from thefirst user's (e.g., user 106A) perspective in FIG. 3. The first user(e.g., user 106A) is not in view of the spatial region 304, but is stillpresent in the mixed reality environment (e.g., the first user (e.g.,user 106A) would be to the left of the dog 308 if pictured).

In the example illustrated in FIG. 4, the third user (e.g., user 106C)can view various virtual content items in his mixed reality environmentvia a display 204 of his device (e.g., device 108C). As described below,the third user (e.g., user 106C) can have authenticated his device(e.g., device 108C). Based at least in part on authenticating his device(e.g., device 108C), the rendering module 132 associated with his device(e.g., device 108C) can render the various virtual content items on thedisplay 204 corresponding to his device (e.g., device 108C), as shown inFIG. 4. As a non-limiting example, the third user (e.g., user 106C) canview a graphical representation of a globe 306, a dog 308, and a drone310. The graphical representations are shown in thick black lines toshow that they represent virtual content.

Unlike the first user (e.g., user 106A) in the example in FIG. 3, thethird user (e.g., user 106C) cannot see the hot air balloon 312 or thecalendar 314. In some examples, the third user (e.g., user 106C) may notsee the hot air balloon 312 or the calendar 314 because the third user(e.g., user 106C) does not have permission to view the hot air balloon312 or the calendar 314 (i.e., the owner of the virtual content item hasnot shared the hot air balloon 312 or the calendar 314 with the thirduser (e.g., user 106C)). Accordingly, the hot air balloon 312 and thecalendar 314 are not visible to the third user (e.g., user 106C). Inother examples, the third user (e.g., user 106C) may not be able to seethe hot air balloon 312 or the calendar 314 because the virtual contentitems are outside of the spatial region 304 within which the third user(e.g., user 106C) can view the mixed reality environment. The third user(e.g., user 106C) can view the globe 306, dog 308, and/or drone 310and/or can interact with the globe 306, dog 308, and/or drone 310 untilthe owner of the virtual content items decides to make the globe 306,dog 308, and/or drone 310 private.

Device/Server: Example Architecture and Processes

FIG. 5 is a schematic diagram showing an example environment 500 forenabling two or more users (e.g., user 106A, user 106B, and/or user106C) in a mixed reality environment to interact with one another and/orwith virtual content that is presented in the mixed reality environment.In the example illustrated in FIG. 5, a first device (e.g., device 108A)is assigned a server role and is responsible for synchronizingcommunications and/or virtual content rendering among all of the devices(e.g., device 108A, device 108B, and/or device 108C). In at least oneexample, devices (e.g., device 108B and/or device 108C) can run anapplication 134 locally and connect to the serving device (e.g., device108A). In FIG. 5, the input module 116, content database 118, contentmanagement module 120, and positioning module 124 can be associated withcomputer-readable media 130 instead of, or in addition to,computer-readable media 114 associated with the service provider 102.

FIG. 5 illustrates a second device (e.g., device 108B) sendingauthentication data to the first device (e.g., device 108A). Theauthentication data can correspond to a user identification and passwordassociated with the second user (e.g., user 106B), biometricidentification associated with the second user (e.g., user 106B), etc.The authentication data can be utilized to determine a presence of thesecond device (e.g., device 108B), visual content items that areavailable to the second user (e.g., user 106B), and the second user's(e.g., user 106B) permissions corresponding to whether the second user(e.g., user 106B) can view and/or interact with individual ones of thevirtual content items, as described above.

Based at least in part on receiving the authentication data, the contentmanagement module 120 and/or the frame rendering module 122 can accesscontent data from the content database 118. As described above, dataassociated with the individual virtual content items can be stored inthe content database 118. The content data may identify an owner of avirtual content item, an identification of the virtual content item, andpermissions associated with the virtual content item. The content datacan identify virtual content items that are owned by a profilecorresponding to the second user (e.g., user 106B), virtual contentitems that the second user (e.g., user 106B) has open, and/or contentthat other users (e.g., user 106A and/or user 106C) have shared with thesecond user (e.g., user 106B). The content management module 120 and/orthe positioning module 124 can utilize the content data to determinewhether individual virtual content items can be rendered on variousdevices (e.g., device 108A and/or device 108C) and/or the permissionsassociated with each of the individual virtual content items. Thecontent management module 120 and/or the positioning module 124 can sendrendering data to the second device (e.g., device 108B) and the seconddevice (e.g., device 108B) can render the corresponding virtual contentitems in the mixed reality environment associated with the second user(e.g., user 106B). As described above, the second user (e.g., user 106B)can modify the permissions (e.g., visibility, interactivity, etc.) ofany of the virtual content items that he or she owns. That is, thesecond user (e.g., user 106B) can share the virtual content items withother users (e.g., user 106A and/or user 106C). However, the second user(e.g., user 106B) cannot modify the permissions of any of the virtualcontent items that he or she does not own. The second user (e.g., user106B) can interact with individual virtual content items until the ownerof each of the virtual content items makes a virtual content itemprivate such that the second user (e.g., user 106B) cannot view thevirtual content item.

The processes described in FIGS. 6 and 7 below are illustrated as acollection of blocks in a logical flow graph, which represent a sequenceof operations that can be implemented in hardware, software, or acombination thereof. In the context of software, the blocks representcomputer-executable instructions stored on one or more computer-readablestorage media that, when executed by one or more processors, perform therecited operations. Generally, computer-executable instructions includeroutines, programs, objects, components, data structures, and the likethat perform particular functions or implement particular abstract datatypes. The order in which the operations are described is not intendedto be construed as a limitation, and any number of the described blockscan be combined in any order and/or in parallel to implement theprocesses. The example processes are described in the context of theenvironment 500 of FIG. 5 but are not limited to that environment.

FIG. 6 is a flow diagram that illustrates an example process 600 tocause virtual content to be presented in the mixed reality environment.

Block 602 illustrates accessing, receiving, and/or determiningauthentication information from a device (e.g., device 108B). Asillustrated in FIG. 5, a second device (e.g., device 108B) can sendauthentication data to a first device (e.g., device 108A) that can bedesignated as a server. The authentication data can correspond to a useridentification and password associated with the second user (e.g., user106B), biometric identification associated with the second user (e.g.,user 106B), etc. The authentication data can be utilized to determine apresence of the second device (e.g., device 108B), visual content itemsthat are available to the second user (e.g., user 106B), and the seconduser's (e.g., user 106B) permissions corresponding to whether the seconduser (e.g., user 106B) can view and/or interact with the virtual contentitems, as described above.

Block 604 illustrates accessing content data. Based at least in part onreceiving the authentication data, content management module 120 and/orpositioning module 124 can access content data from the content database118. As described above, the content database 118 can include contentdata indicating an owner identification, a content identification, andpermissions associated with the individual virtual content items. Thecontent data can identify virtual content items that are owned by aprofile corresponding to the second user (e.g., user 106B) and/orcontent that other users (e.g., user 106A and/or user 106C) have sharedwith the second user (e.g., user 106B).

Block 606 illustrates sending rendering data to the device (e.g., device108B). The content management module 120 and/or positioning module 124can send the rendering data to the second device (e.g., device 108B). Asdescribed above, rendering data may include instructions for rendering agraphical representation of a virtual content item via a display of adevice (e.g., device 108A). For instance, the rendering data may includeinstructions describing the geometry, viewpoint, texture, lighting,shading, etc. associated with a virtual content item.

Block 608 illustrates causing virtual content to be rendered via thedevice (e.g., device 108B). The second device (e.g., device 108B) canleverage the rendering data to render virtual content items in the mixedreality environment associated with the second user (e.g., user 106B)via the rendering module 132 associated with the second device (e.g.,device 108B). As described above, the second user (e.g., user 106B) canmodify the permissions (e.g., visibility, interactivity, etc.) of any ofthe virtual content items that he or she owns. That is, the second user(e.g., user 106B) can determine who he or she desires to share thevirtual content items with and/or permissions the other users (e.g.,user 106A and/or user 106C) have with respect to interacting with thevirtual content items. However, the second user (e.g., user 106B) cannotmodify the permissions of any of the virtual content items that he orshe does not own. The second user (e.g., user 106B) can interact withindividual virtual content items until the owner of each of the virtualcontent items makes a virtual content item private such that the seconduser (e.g., user 106B) cannot view the virtual content item.

FIG. 7 is a flow diagram that illustrates an example process 700 tocause virtual content to be presented in the mixed reality environmentin different modes (e.g., presenter mode or sharing mode).

Block 702 illustrates causing virtual content to be rendered via adevice (e.g., device 108A, device 108B, and/or device 108C). Asdescribed above, the rendering module 132 associated with the device(e.g., device 108A, device 108B, and/or device 108C) can render virtualcontent items corresponding to the rendering data in the mixed realityenvironment associated with the user (e.g., user 106A, user 106B, and/oruser 106C).

Block 704 illustrates determining a mode for presenting the virtualcontent. In some examples, the content management module 120 candetermine whether a user (e.g., user 106A) desires to present thevirtual content in a sharing mode, whereby other users (e.g., user 106Band/or user 106C) can view virtual content items that the user (e.g.,user 106A) shared with them via an enhanced user interface such that thevirtual content items augment the real scene where the other users(e.g., user 106B and/or user 106C) are physically located within thespatial region, or a presentation mode. The presentation mode can enablethe user (e.g., user 106A) to share all of the virtual content that theuser (e.g., user 106A) has open (i.e., is visible on the user's (e.g.,user 106A) display 204) with the other users (e.g., user 106B and/oruser 106C). In such an example, the user (e.g., user 106A) can sharemenus, wands, virtual content items, etc. That is, the presentation modecan enable the user (e.g., user 106A) to share all of the content he orshe has open with all of the other users (e.g., user 106B and/or user106C) (i.e., make all virtual content items public) and share menus thatthe user (e.g., user 106A) generally can see in his or her first personview.

Block 706 illustrates causing the virtual content to be presented insharing mode. As described above, the owner of a virtual content itemcan determine who he or she wants to share the virtual content itemwith. In at least one example, the owner of the virtual content item caninteract with a virtual menu presented to the user (e.g., user 106B) inthe mixed reality environment. As described above, the virtual menu caninclude graphical representations of sharing settings, such as, but notlimited to, private, public, etc. The virtual menu can be a drop downmenu, a radial menu, etc. The virtual menu can provide the user (e.g.,user 106B) with controls for indicating whether the user (e.g., user106B) desires to make the virtual content item private or public, asdescribed above. If the user (e.g., user 106B) desires to keep thevirtual content item private, no other users (e.g., user 106A and/oruser 106C) can see the virtual content item. If the user (e.g., user106B) desires to make the virtual content item public, all of the otherusers (e.g., user 106A and/or user 106C) can see the virtual contentitem. In some examples, the user (e.g., user 106B) can specify one ormore users (i.e., less than all users) with whom he or she desires toshare the virtual content item. This permissions data can be provided tothe content database 118 for storing with the content data. Accordingly,based at least in part on accessing, receiving, and/or determiningauthentication data, the content management module 120 and/or thepositioning module 124 can access content data including permissionsdata indicating which users 106 have permission to view and/or interactwith individual content data items. The other devices (e.g., device 108Aand/or device 108C) can render virtual content items based at least inpart on receiving rendering data from the content management module 120and/or the positioning module 124. The other users (e.g., user 106Aand/or user 106C) can view the rendered virtual content items in theirown first person perspective.

Block 708 illustrates causing the virtual content to be presented inpresentation mode. In other examples, the content management module 120can determine that a user (e.g., user 106B) desires to share his or hermixed reality environment with other users (e.g., user 106A and/or user106C) in a presenter mode. Accordingly, the content management module120 and/or the positioning module 124 can send rendering data to devicesassociated with the other users (e.g., device 108A and/or device 108C)such that the corresponding rendering modules 132 can render virtualcontent consistent with the virtual content presented in the user's(e.g., user 106B) mixed reality environment. The presenter mode enablesthe user (e.g., user 106B) to show other users (e.g., user 106A and/oruser 106C) how to use an application or give a demonstration of thesystem. Presenter mode is similar to various desktop sharingfunctionalities. The other users (e.g., user 106A and/or user 106C) cansee the user's (e.g., user 106B) mixed reality environment from theirown first person perspective. In at least one example, the other users(e.g., user 106A and/or user 106C) can see their private mixed realityenvironment in addition to the mixed reality environment of the user(e.g., user 106B).

Service Provider/Server: Example Architecture and Processes

FIG. 8 is a schematic diagram showing an example environment 800 forenabling two or more users (e.g., user 106A, user 106B, and/or user106C) in a mixed reality environment to interact with one another and/orwith virtual content that is presented in the mixed reality environment.In FIG. 8, the service provider 102 serves a server role and isresponsible for synchronizing communication and/or virtual contentrendering by the devices (e.g., device 108A, device 108B, and/or device108C). The devices (e.g., devices 108A, device 108B, and/or device 108C)can run an application 134 locally and receive frame messages and/orframes for presenting the virtual content.

FIG. 8 illustrates a device (e.g., device 108A) sending authenticationdata to the service provider 102. The authentication data can correspondto a user identification and password associated with the user (e.g.,user 106A), biometric identification associated with the user (e.g.,user 106A), etc. The authentication data can be utilized to determine apresence of the device (e.g., device 108A), visual content that isavailable to the user (e.g., user 106A), and the user's (e.g., user106A) permissions corresponding to whether the user (e.g., user 106A)can view and/or interact with the virtual content, as described above.

Additionally, the device (e.g., device 108A) can send frame requestmessages in real time, as described above. The frame rendering module122 can be configured to generate frame messages responsive to the framerequests, as described above. The frame rendering module 122 can sendframe messages directly to devices 108.

Based at least in part on receiving the authentication data, the contentmanagement module 120 and/or the positioning module 124 can accesscontent data from the content database 118. As described above,individual virtual content items can be associated with data in thecontent database 118 indicating an owner identification, a contentidentification, and permissions associated with the individual virtualcontent items.

Unlike the example environment 500, in this example, the serviceprovider 102 cannot simply send content data to the device (e.g., device108A) and the device (e.g., device 108A) cannot simply render thevirtual content in screenspace as the presentation of the virtualcontent can be affected by noticeable latency (e.g., movement of a user(e.g., user 106A) and/or device (e.g., device 108A) that happened beforea frame is rendered on the device (e.g., device 108A)). Instead, in atleast one example, the rendering module 132, stored locally on thedevice (e.g., device 108A), can utilize the frame messages for renderingand/or presenting the virtual content via the device (e.g., device108A), as described above.

The process described in FIG. 9 below is illustrated as a collection ofblocks in a logical flow graph, which represent a sequence of operationsthat can be implemented in hardware, software, or a combination thereof.In the context of software, the blocks represent computer-executableinstructions stored on one or more computer-readable storage media that,when executed by one or more processors, perform the recited operations.Generally, computer-executable instructions include routines, programs,objects, components, data structures, and the like that performparticular functions or implement particular abstract data types. Theorder in which the operations are described is not intended to beconstrued as a limitation, and any number of the described blocks can becombined in any order and/or in parallel to implement the processes. Theexample process is described in the context of the environment 800 ofFIG. 8 but is not limited to that environment.

FIG. 9 is a flow diagram that illustrates an example process 900 tocause virtual content to be presented in the mixed reality environment.

Block 902 illustrates accessing, receiving, and/or determiningauthentication information from a device (e.g., device 108A). Asillustrated in FIG. 8, the input module 116 can access, receive, and/ordetermine authentication data from a device (e.g., device 108A). Theauthentication data can correspond to a user identification and passwordassociated with the user (e.g., user 106A), biometric identificationassociated with the user (e.g., user 106A), etc. The authentication datacan be utilized to determine a presence of a device (e.g., device 108A),visual content that is available to the user (e.g., user 106A)corresponding to the device (e.g., device 108A), and the user's (e.g.,user 106A) permissions corresponding to whether the user (e.g., user106A) can view and/or interact with the virtual content, as describedabove.

Block 904 illustrates receiving a frame request from the device (e.g.,device 108B). The frame rendering module 122 can receive frame requestmessages from the device (e.g., device 108A), as described above.

Block 906 illustrates accessing content data. Based at least in part onreceiving the authentication data, the content management module 120and/or the positioning module 124 can access content data from thecontent database 118. As described above, content data can be stored inthe content database 118 indicating an owner identification, a contentidentification, and permissions associated with the individual virtualcontent items.

Block 908 illustrates accessing frame messages. The frame renderingmodule 122 can be configured to output frame messages and send the framemessages directly to devices 108, as described above.

Block 910 illustrates sending rendering data and/or frame messages tothe device (e.g., device 108A). In some examples, the rendering module132 can receive previously rendered frames associated with framemessages from the service provider 102 to correct for potential latencyand/or render correct perspectives based on the position of the user(e.g., user 106A) in worldspace. In other examples, the rendering module132 may receive rendering data for rendering the virtual content itemslocally.

Block 912 illustrates causing virtual content to be presented via thedevice (e.g., device 108A). The device (e.g., device 108A) can rendervirtual content items corresponding to rendering data in the mixedreality environment associated with the user (e.g., user 106A) via therendering module 132 associated with the device (e.g., device 108A). Asdescribed above, in some instances, the service provider 102 may beunable to send rendering data to the device (e.g., device 108A) and thedevice (e.g., device 108A) may be unable to render the virtual contentin screenspace as the presentation of the virtual content can beaffected by noticeable latency. Instead, in at least one example, therendering module 132, stored locally on the device (e.g., device 108A),can leverage the frame messages to cause the virtual content items to bepresented via the device (e.g., device 108A), as described above.

Modifying Visibility and Interacting with Virtual Content Items

FIG. 10 is a schematic diagram showing an example environment 1000 forenabling two or more users (e.g., user 106A, user 106B, and/or user106C) in a mixed reality environment to interact with one another and/orwith virtual content that is presented in the mixed reality environment.In the example environment 1000, a server 1002 can send and receive datato one or more client devices (e.g., client 1004, client 1006, and/orclient 1008). The server 1002 can be the service provider 102 or adevice (e.g., device 108A). Each of the client devices (e.g., client1004, client 1006, and/or client 1008) can correspond to device 108A,108B, and/or 108C. The one or more client devices (e.g., client 1004,client 1006, and/or client 1008) can send requests and receive dataassociated with commands for rendering and/or interacting with virtualcontent. The server 1002 can manage the requests and commands tosynchronize communication and/or virtual content rendering between theone or more client devices (e.g., client 1004, client 1006, and/orclient 1008) and to support security, syncing of variables (e.g., statevariables), managing timing (e.g., animation timing, etc.), etc.

The processes described in FIGS. 11 and 12 below are illustrated as acollection of blocks in a logical flow graph, which represent a sequenceof operations that can be implemented in hardware, software, or acombination thereof. In the context of software, the blocks representcomputer-executable instructions stored on one or more computer-readablestorage media that, when executed by one or more processors, perform therecited operations. Generally, computer-executable instructions includeroutines, programs, objects, components, data structures, and the likethat perform particular functions or implement particular abstract datatypes. The order in which the operations are described is not intendedto be construed as a limitation, and any number of the described blockscan be combined in any order and/or in parallel to implement theprocesses. The example process is described in the context of theenvironment 1000 of FIG. 10 but is not limited to that environment.

FIG. 11 is a flow diagram that illustrates an example process 1100 tocause the visibility of virtual content to be modified in the mixedreality environment.

Block 1102 illustrates receiving, from a first client device (e.g.,client 1004), a request to share a virtual content item with one or moreother client devices (e.g., client 1006 and/or client 1008). In thisexample, the first client device (e.g., client 1004) can be the owner ofthe virtual content item and can determine who to share the virtualcontent item with and/or when to make the virtual content item private.

Block 1104 illustrates sending a command to one or more other clientdevices (e.g., client 1006 and/or client 1008). The server 1002 can sendthe command to the one or more other client devices (e.g., client 1006and/or client 1008). The command can include rendering data and, in someexamples, frame messages for rendering by the one or more other clientdevices (e.g., client 1006 and/or client 1008), as described above.

Block 1106 illustrates causing the virtual content item to be presentedvia the one or more other client devices (e.g., client 1006 and/orclient 1008). As described above, the rendering module 132 associatedwith the individual client devices (e.g., client 1006 and/or client1008) can render the virtual content item in the mixed realityenvironment. The users (e.g., user 106A and/or user 106B) correspondingto the one or more other client devices (e.g., client 1006 and/or client1008) can interact with the virtual content item as long as the firstclient device (e.g., client 1002) continues to share the virtual contentitem with the one or more other client devices (e.g., client 1006 and/orclient 1008). In some examples, the one or more client devices (e.g.,client 1006 and/or client 1008) may receive previously rendered framesvia frame messages and may present the virtual content item based atleast in part on presenting the previously rendered frames.

The first client device (e.g., client 1004) can request to make thevirtual content item private via a process similar to example process1100. The first client device (e.g., client 1004) can request to make avirtual content item private with respect to one or more other clientdevices (e.g., client 1006 and/or client 1008). The server 1002 can senda command to one or more other client devices (e.g., client 1006 and/orclient 1008). The command can include data indicating that the virtualcontent item is no longer visible with respect to one or more of theother client devices (e.g., client 1006 and/or client 1008), asdescribed above. Accordingly, the rendering module 132 associated withthe individual client devices (e.g., client 1006 and/or client 1008) canterminate rendering the virtual content item in the mixed realityenvironment.

In some examples, based at least in part on any one of the clientdevices (e.g., client 1004, client 1006, and/or client 1008) accessing avirtual content item, the server 1002 can send data associated with avirtual content item to each of the other client devices (e.g., client1004, client 1006, and/or client 1008). The data can instruct all of theclient devices (e.g., client 1004, client 1006, and/or client 1008)(i.e., the rendering module 132 associated with all of the clientdevices) to create and load the virtual content item. However, theclient devices (e.g., client 1004, client 1006, and/or client 1008) canhide the virtual content item until the client devices (e.g., client1004, client 1006, and/or client 1008) receive data indicating thatowner of the virtual content item shared the virtual content item withthe client devices (e.g., client 1004, client 1006, and/or client 1008).The owner of the virtual content item can send a request to share thevirtual content item with one or more of the other client devices (e.g.,client 1004, client 1006, and/or client 1008) and the server 1002 cansend a command to the one or more client devices (e.g., client 1004,client 1006, and/or client 1008) so that the one or more client devices(e.g., client 1004, client 1006, and/or client 1008) can render thevirtual content item. The command may be associated with rendering dataand/or frame messages, as described above.

FIG. 12 is a flow diagram that illustrates an example process 1200 tocause an interaction with a virtual content item to be performed via oneor more client devices (e.g., client 1004, client 1006, and/or client1008) in the mixed reality environment.

Block 1202 illustrates receiving, from a first client device (e.g.,client 1004), a request to interact with a virtual content item. Forinstance, the first client device (e.g., client 1004) can desire to movethe virtual content item, cause the virtual item to rotate, etc. Block1204 illustrates sending a command to all of the client devices (e.g.,client 1004, client 1006 and/or client 1008) that have permission toview the virtual content item to perform the interaction requested. Theserver 1002 can send the command to the client devices (e.g., client1004, client 1006, and/or client 1008) that have permission to view thevirtual content item. The command can include rendering data, dataassociated with the interaction, and, in some examples, frame messagesfor rendering by the client devices (e.g., client 1004, client 1006,and/or client 1008), as described above. Block 1206 illustrates causingthe interaction to be performed on the virtual content item via theclient devices (e.g., client 1004, client 1006, and/or client 1008). Asdescribed above, the rendering module 132 associated with the individualclient devices (e.g., client 1004, client 1006, and/or client 1008) canrender the virtual content item in the mixed reality environment and canmodify the virtual content item based on the interaction requested. Theusers (e.g., user 106A, user 106B, and/or user 106C) corresponding tothe client devices (e.g., client 1004, client 1006, and/or client 1008)can interact with the virtual content item as long as the first clientdevice (e.g., client 1004) continues to share the virtual content itemwith the one or more other client devices (e.g., client 1006 and/orclient 1008).

Example Clauses

A. A system comprising: a mixed reality display device; and a devicecommunicatively coupled to the mixed reality display device, the devicecomprising: one or more processors; memory; and one or more modulesstored in the memory and executable by the one or more processors toperform operations comprising: determining a presence of the mixedreality display device based at least in part on authenticationinformation associated with the mixed reality display device;determining visibility of a plurality of content items for the mixedreality display device based at least in part on: accessing content dataassociated with the plurality of content items, the content dataindicating an identification of individual content items of theplurality of content items, an owner of each of the individual contentitems, and permissions associated with each of the individual contentitems; determining, based at least in part on the authenticationinformation and the content data, a content item of the plurality ofcontent items that is at least one of owned by the mixed reality displaydevice or has been shared with the mixed reality display device; andcausing a graphical representation of the content item to be presentedvia a display on the mixed reality display device.

B. The system as paragraph A recites, the operations further comprising:sending rendering data associated with the content item to the mixedreality display device; and causing the graphical representation of thecontent item to be rendered via the mixed reality display device.

C. The system as paragraph B recites, the operations further comprising,prior to causing the graphical representation of the content item to berendered via the display, receiving a frame request from the mixedreality display device, the frame request including at least one of poseinformation associated with each eye of a user associated with the mixedreality display device, a time stamp, a desired resolution, or a desiredfield of view.

D. The system as paragraph C recites, the operations further comprising:determining a frame message corresponding to the frame request, theframe message including at least one of the pose information associatedwith each eye of the user, the time stamp, a determined resolution, adetermined field of view, or a render distance; sending the framemessage to the mixed reality display device; and causing the graphicalrepresentation of the content item to be rendered via the display basedat least in part on the frame message.

E. The system as paragraph D recites, wherein the frame message furtherincludes rendered stereo images for each eye of the user.

F. The system as any of paragraphs A-E recite, wherein the permissionsindicate a plurality of devices with which the content item has beenshared.

G. The system as any of paragraphs A-F recite, wherein the permissionsindicate interactions that are available for the content item and aplurality of devices permitted to cause the interactions.

H. A mixed reality display device comprising: one or more processors;memory; and one or more modules stored in the memory and executable bythe one or more processors to perform operations comprising: determiningauthentication information associated with a second mixed realitydisplay device; determining that a content item of a plurality ofcontent items is visible in a mixed reality environment associated withthe second mixed reality display device based at least in part on:accessing content data associated with the plurality of content items,the content data indicating an identification of the content item, anowner of the content item, and permissions associated with the contentitem; and determining, based at least in part on the authenticationinformation and the content data, that the content item is at least oneof owned by the second mixed reality display device or has been sharedwith the second mixed reality display device; and causing a graphicalrepresentation of the content item to be rendered via a display on thesecond mixed reality display device.

I. The mixed reality display device as paragraph H recites, theoperations further comprising: determining, based at least in part onthe content data, that the content item is owned by the mixed realitydisplay device; determining that the content item has been made private;and terminating the causing of the graphical representation of thecontent item to be rendered via the display of the second mixed realitydisplay device.

J. The mixed reality display device as paragraph I recites, theoperations further comprising: causing a menu with graphicalrepresentations of sharing settings to be presented via the mixedreality display device, the sharing settings including a private sharingsetting or a public sharing setting; determining selection of theprivate sharing setting; and determining that the content item has beenmade private based at least in part on the selection of the privatesharing setting.

K. The mixed reality display device as any of paragraphs H-J recite, theoperations further comprising: determining, based at least in part onthe content data, that the second mixed reality display device is notthe owner of the content item; determining that the content item is nolonger being shared with the second mixed reality display device; andterminating the causing of the graphical representation of the contentitem to be rendered via the display on the second mixed reality displaydevice.

L. The mixed reality display device as paragraph K recites, wherein:determining that the content item is visible is further based at leastin part on sharing settings associated with the content item; and theoperations further comprise: determining, based at least in part on thecontent data, that a third mixed reality display device is the owner ofthe content item; receiving a request from the third mixed realitydisplay device to make the content item private; changing the sharingsettings based at least in part on the request; and determining that thecontent item is no longer being shared with the second mixed realitydisplay device based at least in part on changes to the sharingsettings.

M. The mixed reality display device as any of paragraphs H-L recite, theoperations further comprising: determining authentication informationassociated with a third mixed reality display device; determining, basedat least in part on the authentication information associated with thethird mixed reality display device and the content data, that thecontent item is not owned by the third mixed reality display device andhas not been shared with the third mixed reality display device; andprohibiting the graphical representation of the content item from beingrendered via a display on the third mixed reality display device.

N. The mixed reality display device as any of paragraphs H-M recite, theoperations further comprising: receiving, from the owner of the contentitem, a request to share the content item with at least one additionalmixed reality display device; sending rendering data to the at least oneadditional mixed reality display device; and causing the graphicalrepresentation of the content item to be rendered via a display on theat least one additional mixed reality display device.

O. A method for managing permissions of a plurality of content itemsassociated with mixed reality display devices, the method comprising:determining authentication information associated with a first mixedreality display device of the mixed reality display devices; receiving,from the first mixed reality display device, a first request to share acontent item of the plurality of content items with one or more secondmixed reality display devices of the mixed reality display devices;accessing content data associated with the plurality of content items,the content data indicating at least one of an identification ofindividual content items of the plurality of content items, an owner ofeach of the individual content items, or permissions associated witheach of the individual content items; determining, based at least inpart on the authentication information and the content data, that thefirst mixed reality display device has permission to share the contentitem; and causing a graphical representation of the content item to bepresented via the one or more second mixed reality display devices.

P. The method as paragraph O recites, further comprising: receiving,from a second mixed reality display device of the one or more secondmixed reality display devices, a second request associated with aninteraction affecting the content item; determining, based at least inpart on the content data, that the second mixed reality display deviceis permitted to interact with the content item via the interaction; andsending a command to the first mixed reality display device and othermixed reality display devices of the one or more second mixed realitydisplay devices to perform the interaction.

Q. The method as paragraphs O or P recite, further comprising receiving,from a requesting device comprising at least one of the first mixedreality display device or a second mixed reality display device of theone or more second mixed reality display devices, a frame requestincluding at least one of pose information associated with each eye of auser corresponding to the requesting device, a time stamp, a desiredresolution, or a desired field of view.

R. The method as paragraph Q recites, further comprising: determining aframe message corresponding to the frame request, the frame messageincluding at least one of the pose information associated with each eyeof the user, the time stamp, a determined resolution, a determined fieldof view, or a render distance; sending the frame message to therequesting device; and causing the graphical representation of thecontent item to be presented via the first mixed reality display deviceand the one or more second mixed reality display devices based at leastin part on the frame message.

S. The method as paragraphs O-R recite, further comprising: sending atleast one of rendering data or rendered stereo images to the first mixedreality display device and the one or more second mixed reality displaydevices; and causing the graphical representation of the content item tobe presented via the first mixed reality display device and the one ormore second mixed reality display devices based at least in part on atleast one of the rendering data or the rendered stereo images.

T. The method as paragraphs O-S recite, further comprising: receiving,from the first mixed reality display device, a second request to share amixed reality environment associated with the first mixed realitydisplay device with the one or more second mixed reality displaydevices, the mixed reality environment including the content item andone or more additional content items; and causing the mixed realityenvironment to be presented via the one or more second mixed realitydisplay devices to enable a presenter mode.

U. The method as paragraphs O-T recite, wherein the permissions indicatea plurality of devices with which the content item has been shared.

V. The method as paragraphs O-U recite, wherein the permissions indicateinteractions that are available for the content item and a plurality ofdevices permitted to cause the interactions.

W. One or more computer-readable media encoded with instructions that,when executed by a processor, configure a computer to perform a methodas any of paragraphs O-V recite.

X. A device comprising one or more processors and one or more computerreadable media encoded with instructions that, when executed by the oneor more processors, configure a computer to perform acomputer-implemented method as any of paragraphs O-V recite.

Y. A method for managing permissions of a plurality of content itemsassociated with mixed reality display devices, the method comprising:means for determining authentication information associated with a firstmixed reality display device of the mixed reality display devices; meansfor receiving, from the first mixed reality display device, a firstrequest to share a content item of the plurality of content items withone or more second mixed reality display devices of the mixed realitydisplay devices; means for accessing content data associated with theplurality of content items, the content data indicating at least one ofan identification of individual content items of the plurality ofcontent items, an owner of each of the individual content items, orpermissions associated with each of the individual content items; meansfor determining, based at least in part on the authenticationinformation and the content data, that the first mixed reality displaydevice has permission to share the content item; and means for causing agraphical representation of the content item to be presented via the oneor more second mixed reality display devices.

Z. The method as paragraph Y recites, further comprising: means forreceiving, from a second mixed reality display device of the one or moresecond mixed reality display devices, a second request associated withan interaction affecting the content item; means for determining, basedat least in part on the content data, that the second mixed realitydisplay device is permitted to interact with the content item via theinteraction; and means for sending a command to the first mixed realitydisplay device and other mixed reality display devices of the one ormore second mixed reality display devices to perform the interaction.

AA. The method as paragraphs Y or Z recite, further comprising means forreceiving, from a requesting device comprising at least one of the firstmixed reality display device or a second mixed reality display device ofthe one or more second mixed reality display devices, a frame requestincluding at least one of pose information associated with each eye of auser corresponding to the requesting device, a time stamp, a desiredresolution, or a desired field of view.

AB. The method as paragraph AA recites, further comprising: means fordetermining a frame message corresponding to the frame request, theframe message including at least one of the pose information associatedwith each eye of the user, the time stamp, a determined resolution, adetermined field of view, or a render distance; means for sending theframe message to the requesting device; and means for causing thegraphical representation of the content item to be presented via thefirst mixed reality display device and the one or more second mixedreality display devices based at least in part on the frame message.

AC. The method as paragraphs Y-AB recite, further comprising: means forsending at least one of rendering data or rendered stereo images to thefirst mixed reality display device and the one or more second mixedreality display devices; and means for causing the graphicalrepresentation of the content item to be presented via the first mixedreality display device and the one or more second mixed reality displaydevices based at least in part on at least one of the rendering data orthe rendered stereo images.

AD. The method as paragraphs Y-AC recite, further comprising: means forreceiving, from the first mixed reality display device, a second requestto share a mixed reality environment associated with the first mixedreality display device with the one or more second mixed reality displaydevices, the mixed reality environment including the content item andone or more additional content items; and means for causing the mixedreality environment to be presented via the one or more second mixedreality display devices to enable a presenter mode.

AE. The method as paragraphs Y-AD recite, wherein the permissionsindicate a plurality of devices with which the content item has beenshared.

AF. The method as paragraphs Y-AE recite, wherein the permissionsindicate interactions that are available for the content item and aplurality of devices permitted to cause the interactions.

CONCLUSION

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described. Rather,the specific features and acts are described as illustrative forms ofimplementing the claims.

Conditional language such as, among others, “can,” “could,” “might” or“can,” unless specifically stated otherwise, are understood within thecontext to present that certain examples include, while other examplesdo not necessarily include, certain features, elements and/or steps.Thus, such conditional language is not generally intended to imply thatcertain features, elements and/or steps are in any way required for oneor more examples or that one or more examples necessarily include logicfor deciding, with or without input or prompting, whether certainfeatures, elements and/or steps are included or are to be performed inany particular example. Conjunctive language such as the phrase “atleast one of X, Y or Z,” unless specifically stated otherwise, is to beunderstood to present that an item, term, etc. can be either X, Y, or Z,or a combination thereof.

What is claimed:
 1. A system comprising: a mixed reality display device;and a device communicatively coupled to the mixed reality displaydevice, the device comprising: one or more processors; memory; and oneor more modules stored in the memory and executable by the one or moreprocessors to perform operations comprising: determining a presence ofthe mixed reality display device based at least in part onauthentication information associated with the mixed reality displaydevice; determining visibility of a plurality of content items for themixed reality display device based at least in part on: accessingcontent data associated with the plurality of content items, the contentdata indicating an identification of individual content items of theplurality of content items, an owner of each of the individual contentitems, and permissions associated with each of the individual contentitems; determining, based at least in part on the authenticationinformation and the content data, a content item of the plurality ofcontent items that is at least one of owned by the mixed reality displaydevice or has been shared with the mixed reality display device; andcausing a graphical representation of the content item to be presentedvia a display on the mixed reality display device.
 2. The system asclaim 1 recites, the operations further comprising: sending renderingdata associated with the content item to the mixed reality displaydevice; and causing the graphical representation of the content item tobe rendered via the mixed reality display device.
 3. The system as claim2 recites, the operations further comprising, prior to causing thegraphical representation of the content item to be rendered via thedisplay, receiving a frame request from the mixed reality displaydevice, the frame request including at least one of pose informationassociated with each eye of a user associated with the mixed realitydisplay device, a time stamp, a desired resolution, or a desired fieldof view.
 4. The system as claim 3 recites, the operations furthercomprising: determining a frame message corresponding to the framerequest, the frame message including at least one of the poseinformation associated with each eye of the user, the time stamp, adetermined resolution, a determined field of view, or a render distance;sending the frame message to the mixed reality display device; andcausing the graphical representation of the content item to be renderedvia the display based at least in part on the frame message.
 5. Thesystem as claim 4 recites, wherein the frame message further includesrendered stereo images for each eye of the user.
 6. The system as claim1 recites, wherein the permissions indicate a plurality of devices withwhich the content item has been shared.
 7. The system as claim 1recites, wherein the permissions indicate interactions that areavailable for the content item and a plurality of devices permitted tocause the interactions.
 8. A mixed reality display device comprising:one or more processors; memory; and one or more modules stored in thememory and executable by the one or more processors to performoperations comprising: determining authentication information associatedwith a second mixed reality display device; determining that a contentitem of a plurality of content items is visible in a mixed realityenvironment associated with the second mixed reality display devicebased at least in part on: accessing content data associated with theplurality of content items, the content data indicating anidentification of the content item, an owner of the content item, andpermissions associated with the content item; and determining, based atleast in part on the authentication information and the content data,that the content item is at least one of owned by the second mixedreality display device or has been shared with the second mixed realitydisplay device; and causing a graphical representation of the contentitem to be rendered via a display on the second mixed reality displaydevice.
 9. The mixed reality display device as claim 8 recites, theoperations further comprising: determining, based at least in part onthe content data, that the content item is owned by the mixed realitydisplay device; determining that the content item has been made private;and terminating the causing of the graphical representation of thecontent item to be rendered via the display of the second mixed realitydisplay device.
 10. The mixed reality display device as claim 9 recites,the operations further comprising: causing a menu with graphicalrepresentations of sharing settings to be presented via the mixedreality display device, the sharing settings including a private sharingsetting or a public sharing setting; determining selection of theprivate sharing setting; and determining that the content item has beenmade private based at least in part on the selection of the privatesharing setting.
 11. The mixed reality display device as claim 8recites, the operations further comprising: determining, based at leastin part on the content data, that the second mixed reality displaydevice is not the owner of the content item; determining that thecontent item is no longer being shared with the second mixed realitydisplay device; and terminating the causing of the graphicalrepresentation of the content item to be rendered via the display on thesecond mixed reality display device.
 12. The mixed reality displaydevice as claim 11 recites, wherein: determining that the content itemis visible is further based at least in part on sharing settingsassociated with the content item; and the operations further comprise:determining, based at least in part on the content data, that a thirdmixed reality display device is the owner of the content item; receivinga request from the third mixed reality display device to make thecontent item private; changing the sharing settings based at least inpart on the request; and determining that the content item is no longerbeing shared with the second mixed reality display device based at leastin part on changes to the sharing settings.
 13. The mixed realitydisplay device as claim 8 recites, the operations further comprising:determining authentication information associated with a third mixedreality display device; determining, based at least in part on theauthentication information associated with the third mixed realitydisplay device and the content data, that the content item is not ownedby the third mixed reality display device and has not been shared withthe third mixed reality display device; and prohibiting the graphicalrepresentation of the content item from being rendered via a display onthe third mixed reality display device.
 14. The mixed reality displaydevice as claim 8 recites, the operations further comprising: receiving,from the owner of the content item, a request to share the content itemwith at least one additional mixed reality display device; sendingrendering data to the at least one additional mixed reality displaydevice; and causing the graphical representation of the content item tobe rendered via a display on the at least one additional mixed realitydisplay device.
 15. A method comprising: determining authenticationinformation associated with a first mixed reality display device;receiving, from the first mixed reality display device, a first requestto share a content item of a plurality of content items with one or moresecond mixed reality display devices; accessing content data associatedwith the plurality of content items, the content data indicating atleast one of an identification of individual content items of theplurality of content items, an owner of each of the individual contentitems, or permissions associated with each of the individual contentitems; determining, based at least in part on the authenticationinformation and the content data, that the first mixed reality displaydevice has permission to share the content item; and causing a graphicalrepresentation of the content item to be presented via the one or moresecond mixed reality display devices.
 16. The method as claim 15recites, further comprising: receiving, from a second mixed realitydisplay device of the one or more second mixed reality display devices,a second request associated with an interaction affecting the contentitem; determining, based at least in part on the content data, that thesecond mixed reality display device is permitted to interact with thecontent item via the interaction; and sending a command to the firstmixed reality display device and other mixed reality display devices ofthe one or more second mixed reality display devices to perform theinteraction.
 17. The method as claim 15 recites, further comprisingreceiving, from a requesting device comprising at least one of the firstmixed reality display device or a second mixed reality display device ofthe one or more second mixed reality display devices, a frame requestincluding at least one of pose information associated with each eye of auser corresponding to the requesting device, a time stamp, a desiredresolution, or a desired field of view.
 18. The method as claim 17recites, further comprising: determining a frame message correspondingto the frame request, the frame message including at least one of thepose information associated with each eye of the user, the time stamp, adetermined resolution, a determined field of view, or a render distance;sending the frame message to the requesting device; and causing thegraphical representation of the content item to be presented via thefirst mixed reality display device and the one or more second mixedreality display devices based at least in part on the frame message. 19.The method as claim 15 recites, further comprising: sending at least oneof rendering data or rendered stereo images to the first mixed realitydisplay device and the one or more second mixed reality display devices;and causing the graphical representation of the content item to bepresented via the first mixed reality display device and the one or moresecond mixed reality display devices based at least in part on at leastone of the rendering data or the rendered stereo images.
 20. The methodas claim 15 recites, further comprising: receiving, from the first mixedreality display device, a second request to share a mixed realityenvironment associated with the first mixed reality display device withthe one or more second mixed reality display devices, the mixed realityenvironment including the content item and one or more additionalcontent items; and causing the mixed reality environment to be presentedvia the one or more second mixed reality display devices to enable apresenter mode.